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This outreach project is based on the validated Developmental Therapy-Teaching model originally designed 
for young children with severe emotional/behavioral problems and for their families. It is an approach that 
emphasizes teaching skills which foster a child's social-emotional-behavioral competence. The model has 
proven effective in inclusive settings with children who are socially, emotionally, or behaviorally delayed. It is 
also effective with children who have learning disabilities, autism, or language delays. Components of the 
model can be integrated into other early childhood preacademic curriculums, and include family involvement 
such as parental participation in assessment, program planning, and simultaneous home implementation of 
model practices as integral aspects of the model. 

The project assists early childhood and local child care programs in replicating components of the 
model in inclusive or pull-out settings for children (from birth to age 8) with social-emotional-behavioral 
disabilities, including those with autism or developmental delay in social-emotional-behavioral development -- 
and their families. Recognizing that effective implementation depends upon the knowledge and skills of the 
adults who work directly with these young children, the project assists participants in understanding behaviors 
and acquiring specific skills to foster social-emotional-behavioral growth of these children. Emphasis is on 
model applications in typical daily activities such as social play, social language, listening and responding, 
creating, imagining, playing, and participating. 

Specific project activities include: (1) Dissemination of information (2) site development activities, with 
outreach and inservice planning for introduction of the model as an addition to existing curriculum; (3) model 
implementation at selected replication sites, adapted to needs of participants and children at each site; and (4) 
topical workshops and general training at workshops and professional meetings for early childhood 
professionals, paraprofessionals, and families of young children in inclusive settings; (5) regional & state 
agency collaboration for program development and personnel training needs; and (6) product development 
to provide training materials for future inservice outreach via long distance learning, multi-media, and 
interactive materials. 

Project evaluation data indicate that 731 participants at 13 program locations received inservice 
assistance in implementing the model in inclusive early childhood settings. Of these, 7 sites with 281 
participants requested in-depth assistance to become replication sites. Post-training focus groups revealed 
that these participants acquired an expanded understanding of using a developmental framework for planning 
activities to meet individual needs of children. Their responses also indicated increased awareness of 
children’s unique social-emotional-behavioral needs. In post-training observations of 65 participants working 
directly with children at 8 sites, 81 % demonstrated "effective" or better performance skills. Data also show that 
a sample of 28 children at 3 sites made statistically significant improvement in social-emotional-behavioral 
development during a 6-month period while their teachers participated in inservice training. 

The Developmental Therapy-Teaching Programs at the University of Georgia have a network in place 
to continue to disseminate products and to support programs or individuals seeking to replicate this model. 
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IV. GOALS AND OBJECTIVES OF THE PROJECT 






This inservice training project is designed to increase the capacity of professional and 
paraprofessional personnel and families to provide and enhance early intervention 
preschool programs with a social-emotional component for the young children they serve. 
Using the Developmental Therapy-Teaching curriculum model, a new adaptation of a 
previously validated intervention model, the project brings inservice opportunities to non- 
specialized personnel in inclusive early childhood service settings. Specific project goals 
are to: 



1. Increase the understanding of personnel and families about how social- 
emotional competence develops and of practical ways they can promote development of 
prosocial behavior in young children. 

2. Increase the skills of the participants in selecting and using developmentally 
appropriate strategies and activities to enhance children’s social-emotional development. 

3. Provide follow-up and support services to participants for skills maintenance 
in the early intervention environment. 

4. Evaluate the project training model and disseminate results to programs and 
personnel, as requested, across the state and country. 



V. CONCEPTUAL FRAMEWORK FOR THE PROJECT 

The central issue for this inservice project is assisting participants in understanding the 
interactive dynamics of social, emotional, behavioral, and intellectual development of 
young children with disabilities so that developmentally appropriate experiences are 
provided in inclusive settings which foster social-emotional-behavioral growth. The 
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curriculum, a team-based model of comprehensive services, emphasizes strengthening 
the healthy process in every child which works to enhance social-emotional development 
and responsible behavior. 

VI. DESCRIPTION OF THE TRAINING MODEL, ACTIVITIES, 

AND PARTICIPANTS 

This inservice training model assists families and program personnel in using the validated 
Developmental Therapy-Teaching curriculum to help children derive pleasure and 
enrichment from normal childhood experiences. The training encompasses five distinct 
content areas: (1) recognizing and assessing normal milestones for healthy social- 
emotional development; (2) selecting developmental^ appropriate activities and materials 
that enhance social-emotional development; (3) observing and decoding children’s 
behavior for greater understanding of the actions through which they convey their social- 
emotional needs; (4) using growth-producing behavior management strategies to meet 
these needs; and (5) participating in team building activities which enhance program 
effectiveness. 

The project was designed with a dual focus: Component 1 addressed the inservice 
training needs of personnel and families working directly with young children; while 
Component 2 addressed advanced inservice needs of leadership personnel with 
responsibility for the quality of the programs. While the essential content of the 
Developmental Therapy-Teaching curriculum model is necessary for each group of 
participants, the actual project activities differ in the delivery of inservice training. The first 
component utilized the following essential activities for personnel (primarily those who are 
non-specialized, in inclusive settings) and families providing direct service to young 
children: 

• Disseminate information through circulation of materials, workshops, and 
presentations. 

• Assess local needs with questionnaires, e-mail and Fax, correspondence, and 
telephone conferences. 
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• Design inservice training agreements including objectives, content sequences and 
support activities, workshops, and follow-up. 

• Implement inservice agreements and evaluate outcomes, including participants’ 
skills and progress of participating children when possible. 

• Adapt and redesign training materials and media packages to meet expressed needs 
of participants in early childhood settings. 

• Develop new learning strategies for the inservice training model as indicated by 
responses to process evaluation feedback from participants. 

• Provide inservice training credits if requested by individuals. 

• Use evaluation results to assess project effectiveness in accomplishing its purpose and 
goals of expanding and enhancing the skills of adults who influence the lives of young 
children with social-emotional-behavioral disabilities. 

The second component utilized somewhat different activities to provide inservice 
training to preschool leadership personnel who have on-site, day-to-day responsibility to 
guide programs, support personnel and families, coordinate services, and ensure the 
quality of programs to young children. The project activities for these participants were 
highly dependent upon assessed needs of individuals to acquire comprehensive, in-depth 
knowledge and skills in assisting others to effectively implement the Developmental 
Therapy-Teaching curriculum. The following activities were used selectively to conduct this 
second component: 

• Determine individual participant’s needs in order to fulfil leadership responsibilities 
in including a high quality Developmental Therapy-Teaching curriculum among the 
program services to young children with disabilities and their families. 

• Coiiaboratively design and individualize inservice training sequence specifying 
objectives, content, skills to be acquired, type of training and follow-up. 

• implement intensive leadership training through local, on-site tutorials, site-based 
follow-up focused on supervisory needs, and topical workshops teamed with project 
instructors in a collaborative tutorial design. 

• Guide leadership participants in developing a program evaluation plan for the local 
site including training personnel in the reliable use of the evaluation instruments and 
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procedures, with effective classroom follow-up. 

VII. METHODOLOGICAL AND LOGISTICAL PROBLEMS 

There were several unanticipated factors which influenced this inservice project. The 
overall goal of the project was to increase the awareness and knowledge of primarily non- 
special education personnel concerning the social-emotional-behavioral growth of young 
children. This goal was achieved primarily in child care programs, including Head Start, 
therapeutic child development programs, and home child care programs. The difficulties 
encountered in accomplishing project goals actually provide a revealing look at the general 
state of services in the field and difficulties which programs face at the local level. 

One of the primary problems in completing the workscope, as originally designed, 
was the vast turnover of personnel in the local programs. Recognizing that this occupation 
has one of the highest documented turnover rates, steps were taken to encourage staff 
stability within selected sites - for at least one calendar year and, hopefully, for more than 
one year. Before selecting sites several telephone conversations were held with 
administrators which highlighted the needs of the programs and the types of services to 
be offered through the project. For example, in the Central Valley of Washington, where 
four of the sites were located, a pre-training meeting was held with potential program 
participants. This meeting included the early childhood coordinator, teachers and 
paraprofessionals who were already involved in the use of the model (and providing 
demonstration assistance); Head Start teachers, team leaders and assistants; and 
Therapeutic Child Development program teachers, assistants, and administrators. A 
commitment was made by this project and local administrators to provide the training. 
Teachers were excited about the potential for training and follow-up. Yet, in two of the 
sites, pilot teams who were trained together were subsequently split by administrators and 
placed with other staff "because of their expertise." This skewed the pre-post data for the 
project. However, more important in terms of subjective factors was the negative effect on 
staff. Verbal feedback from those involved indicated lowered staff morale as a result of the 
changes. They felt a lack of input in decision-making and frustration that they were unable 
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to follow through in their new settings using strategies and classroom practices which they 
found exciting and effective. By the end of the third year, only one trained member 
remained on staff, and this person was at the assistant level, which gave her little 
opportunity for movement. These problems hampered the completion of the evaluation 
component even though the personnel actually involved with children completed initial 
training and had begun very effective classroom implementation as indicated by initial 
DT/RITS and DTORF-R data. Thus, high local staff turnover became a primary factor in 
documentation of project effects. 

A second significant factor in the completion of the project was the need for local 
coordinators designated to directly communicate with the project staff. The responsibilities 
of these individuals included some oversight of child data and support and supervision of 
local staff. Much of the success of implementation relied on the abilities of this person to 
support the staff and to assist them in recognizing the importance of social-emotional- 
behavioral competence in the overall development of their youngsters. When this attitude 
was fostered by a local administrator and the collection of data was organized in a 
systematic manner, the teams flourished, and data indicated significant gains of children 
and positive attitudes of participants. 

One proposed avenue of follow-up to local programs was not adequately tested 
within the parameters of the project. It had been proposed that video consultation/feedback 
would be available. The plan was that difficult classroom or child care situations would be 
videotaped and sent to the project office for review and conference call feedback. This 
strategy has been used effectively in other settings but was only used with one site in this 
project because of confidentiality limitations. The participating Therapeutic Child 
Development programs enrolled children who were court-placed. Guardians were the 
Department of Children and Family Services. It was the interpretation of the local worker 
in these programs that videos, even for educational purposes within the programs, were 
not permissible. 

Perhaps the most rewarding measures of change are reflected in the individual 
comments of participants. These have been expressed through the focus group interview, 
workshop evaluations, and personal responses. Comments such as "This helped me to 

8 




9 



understand children better than any other training" were common. Comments are included 
in the Attachments at the end of this report. 

VIII. EVALUATION FINDINGS 

The first evaluation question concerned general project outcomes. The question was the 
extent to which the project accomplished its mission, goals, and objectives, with activities 
as originally proposed. Table 1 contains a summary of project outcomes and indicates that 
all of the proposed project activities were implemented and accomplished or 
exceeded projected outcomes. These findings, in brief, indicate that the project originally 
proposed to provide inservice outreach assistance to 10 - 15 sites and 503 service 
providers over the three-year grant period, thus expecting to impact programmatically on 
about 2,100 children. Actual results show that the project provided inservice activities at 
13 different program locations with 88 on-site training days for 731 participants whose 
training impacted approximately 3,655 children with disabilities, (using an estimated 
average of 5 children per participating adult). Because the programs were inclusive by 
design, this figure does not include the larger, beneficial effect provided indirectly to many 
more young children without disabilities in these early childhood programs. In depth, 
extensive outreach was provided to 7 of the 13 locations which requested assistance in 
implementing the model adoption as a replication site. Figure 1 outlines the essential 
requirements for model adoption met by these 7 sites. 



Table 1. Project Outcomes: Summary Data 



Outreach Activity 

Information dissemination 
& awareness 



Projected 

4 states 



Accomplished 

total = 10 States 

Alabama 

Connecticut 

Georgia 

Kentucky 

Minnesota 

Nevada 

South Carolina 

Utah 

Virgin Islands 
Washington State 



Information packages mailed 




total = 1 79 information packages 


DTORF-R assessment instruments mailed 




total = 291 DTORF-R packages 


Newsletters 




total = 768 on mailing list 


Presentations at conferences 




total = 1 8 presentations to approx. 






1 ,335 participants 


Sit^development activities 






# Sites 


1 0-1 5 locations 


total = 13 program locations* 


# Participants 


503 participants 


total = 88 training days 


# Children impacted 


2100 children 


total = 731 participants 




(est.@ 5 


total = 3655 children impacted 




children per 
participant) 


(est. @ 5 children per participant) 


Replication sites developed 




total = 7 replications sites 


site 1 * Learning Tree 
Therapeutic & regular child care 
Bremerton, WA 






Needs assessment 


yes 


yes 


# Training visits 




# training days = 8 


# Participants 


10 


# participants, total = 38 


# Children impacted 


20 


# children = 23 with disabilities 


# Site visit reports 


yes 


# reports = 8 


site 2 • Sunshine & Rainbows 

Therapeutic & regular child care 
Forks. WA 






Needs assessment 


yes 


yes 


# Training visits 




# training days = 4 


# Participants 


12 


# participants, total = 50 


# Children impacted 


20-50 


# children = 15 with disabilities 


# Site visit reports 


yes 


# reports = 5 



* The 13 sites receiving project services include the 7 programs identified as replication sites, plus 2 additional 
locations in the McIntosh Head Start program, 1 additional location in the EPIC Upper Valley Yakima program, 1 
program in Athens, Georgia, 1 program as a "second generation" program under the leadership of a previously 
trained person, and 1 other site (Virgin Islands) which was well underway but destroyed in the '95 hurricane. 
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Table 1 (continued) 



Outreach Activity Projected 



site 3 - EPIC - Upper Valley 

(2 locations) 

Castlevale Child Development 
Yakima, WA 

Needs assessment 


yes 


# Participants 


12 


# Children impacted 


20- 


# Site visit reports 


yes 


site 4 - EPIC - Mid Valley 

Therapeutic Child Development 
Toppenish, WA 

Needs assessment 


yes 


# Participants 


12 


# Children impacted 


20 


# Site visit reports 


yes 


site 5 - EPIC - Lower Valley 

Therapeutic Child Development 
Sunnyside, WA 

Needs assessment 


yes 


# Participants 


12 


# Children impacted 


20- 


# Site visit reports 


yes 


site 6 - McIntosh Head Start 

(3 locations) 

Spaulding, Pike, & Henry 
Counties, GA 

Needs assessment 


yes 


# Participants 


12 


# Children impacted 


30 


# Site visit reports 


yes 


site 7 - Monarch Child Care 

Lacey, WA 
Needs assessment 


yes 


# Participants 


19 


# Children impacted 


20 


# Site visit reports 


yes 



Topical workshops 50 participants 

(Special 1 - day workshops on 

requested topics to groups 

other than potential replications 

sites) 



Accomplished 



yes: # training days = 13 

# participants, total = 84 

# children = 1 2 with disabilities 

# reports = 1 6 



yes; # training days = 14 

# participants, total = 53 

# children = 1 9 with disabilities 

# reports = 9 



yes; # training days - 10 

# participants, total - 26 

# children = 1 2 with disabilities 

# reports = 4 



yes; # training days = 8 

# participants, total = 22 

# children = 30 with disabilities 

# reports = 7 



yes; # training days = 4 

# participants, total = 8 

# children = 20 with disabilities 

# reports = 2 

Total = 13 topical workshops 
Total = 480 participants 
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Outreach Activity 

Regional/state agency 
collaboration 



Product development 

Slide/tape for assessment 
Video training segments 

• developmental^ appropriate 

strategies for positive 
management of difficult 
behavior 

• developmental^ strategic 

activities booklets 



Table 1 (continued) 

Projected Accomplished 

5 agencies total = 9 agencies 

Connecticut Early Childhood Network 
Georgia State Agency Planning & 
Coordination Committee 
Gwinnett Coordinating Agencies 
NEC * TAS 
RAP, Chapel Hill, NC 
Virgin Islands Dept. Of Education 
Washington Educational Service Districts 
113 & 114 

Washington State DSHS 
Washington EPIC Mid-Valley area 



yes yes, final version field-testing 

yes yes, final version field-testing 



collection underway 
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Figure 1. 

^^^^^^^^ ^^^iitiaij : Standards'Metb^Sites-^Replicatin9lhiMod||^^P ^^^^^^^^ 
Several mintmal standards have been designated as necessary for a quality replication;; 

1 . initial staff training in the Developmental Therapy-Teaching curriculum. 

2- litysfdftHepeYelopmental Teaching .Objectives Rating Fbrm (DTQ rItR) with an accuracy score pf 
. 75% correct or better. 

3: i : i Selection of services for children based on IPSP and IEP goals, and utilisation of the model according 
to each child's developmental stage. i£>YV. 

4. Use of the specified classroom practices and procedures as indicated by a classroom performance 
of 75% or better on the Developmental Therapy Rating Inventory of Teacher Skills (DT/RITS). |||| 

5. Involvement ^|||e family and chiid'sfteachers in the DTORF-R 'ratings and evaluation of the 

DeVelopmentalTherapy-Teaching pfogrsmwhenf^^ible. .! 

6. || Provision for concomitant enrollment of each child in inclusive or Integrated educational placement, 

• when possible. • • • ; 

llli An evaluation plan which includes the use of the DjFORF-R and the DT/RITS, reporting data on a pre- 
llllpost basis along with other evaluation data required for annual reports. In addition, sites submit chjjd 
flllll! progress data from at least one standardized instrument, when possible|^||i|p|p|||^ 
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The Evaluation Plan included efforts to determine the extent to which participants 
gained new knowledge and actually implemented this knowledge with increased skill in the 
service settings. Project evaluation was concerned also with the impact on the children 
taught by participants. Because systematic evaluation of staff and children at service 
centers is frequently limited by local policies, project effectiveness was evaluated using 
several different subject-selection procedures: self-selecting volunteers for focus groups 
and written follow-up questionnaires, randomly chosen children from sites where reliable 
child progress data were available, and teacher performance measures where permitted 
by local administrators. To judge project effectiveness with these selected subjects, the 
following evaluation questions were addressed and findings reported below: 

Concerning the Participants Receiving Outreach Inservice Training: 

After the training, do participants demonstrate understanding of the social- 
emotional-behavioral development of young children, the program needs they have, 
and the skills needed by adults to provide what is needed? 

Two measures were used for gathering data to answer this question: a 32 item multiple- 
choice quiz administered directly before and after training, and an on-site focus group of 
volunteer participants discussing seven questions related to the effect of the training on 
their understanding of the model and its content regarding the social-emotional-behavioral 
development of children and their program needs. 

The 32-item paper and pencil test was administered to 155 participants from 13 
sites at the completion of their training. This multiple-choice test included basic knowledge 
of milestones for social-emotional-behavioral development, birth to age 5. As a group, the 
participants achieved an average of 22 items correct (70%). There was also a subset of 
both pre- and post-training data for 1 10 of these participants from 1 1 sites. They achieved 
an average of 20 items correct on the pre-test (range = 4 to 27) and 22 items correct on 
the post-test (range = 10 - 29) - a 12.4% improvement. These findings suggest that the test 
content may have included generally basic information about development that most early 
childhood educators should know prior to inservice training for this model. However, project 
instructors also noted that many participants had expressed discomfort over having to 
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"take a test" and many indicated confusion over words on the test. This suggested that a 
paper and pencil test might be an inappropriate means for measuring practical knowledge 
gained by these participants during the training. 

To further explore this idea, the data were sorted into 3 groups. Group 1 participants 
had completed no more than a high school education or less (n = 19), Group 2 participants 
had some education beyond high school (n = 77) and Group 3 participants did not identify 
their educational level (n = 14). Table 2 summarizes the performance of these three 
groups on the paper-pencil test. 

Table 2. Knowledge Test Scores, Pre- and Post-Training 





N 


Average 

Pre-Test 

Correct 


(Range) 


Average 
Post-T est 
Correct 


(Range) 


Improvement 


Group 1 


19 


17 


(9 - 25) 


19 


(10-26) 


9% 


Group 2 


77 


20 


(4 - 27) 


23 


(10-29) 


1 3.5% 


Group 3 


14 


14 


(13-25) 


15 


(11 -28) 


10% 


Total 


110 


20 


(9 - 27) 


22 


(10-29) 


12.4% 



It is clear that participants with more educational preparation (Group 2) were able 
to handle this written test more successfully than participants with less education. It may 
be that the test covered basic child development content so that those who had some prior 
course work were at a distinct advantage on both pre- and post-training testing. However, 
the data also brings into question the general validity of this form of evaluation. It leaves 
unanswered the question concerning the extent to which these participants’ knowledge (as 
measured by paper and pencil test) increased during the training. 

The focus group format seems to be a more suitable and effective way to assess 
participants’ understanding. Attachment A summarizes the focus group responses for 
sites 1 and 2. These responses were recorded using audio tapes, as video appeared too 
intrusive for relaxed participation. Responses to each question were then summarized, 
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categorized, and ranked with most frequently repeated ideas reported first. The preliminary 
discussion in response to question 1 indicates that participants received varying amounts 
of training - between 18 and 20 hours in total (negotiated individually with each site at the 
time of the needs assessment prior to training). Some of the participants had also received 
information about the model from a college class taught by leadership participant in the 
project. 

All of the responses were positive about the content but conveyed concern about 
time constraints limiting the amount of effort that they can expend to implement exemplary 
practices. In spite of this concern, the responses to question 4 suggest that participants 
believed they had an expanded understanding of using a developmental framework for 
planning program activities to meet individual social-emotional needs and to guide each 
child’s progress in a sequential, targeted direction. Responses to questions 2 and 3 
indicate that participants had acquired understanding of the need to assess each child's 
social-emotional-behavioral development as the first step in establishing individual program 
objectives. Their responses also indicated they had increased awareness of the unique 
social-emotional needs of individual children and had acquired many of the recommended 
strategies to meet these needs. The conclusion to be drawn from these focus group 
discussions is that if time permitted, they would like to complete similar assessments for 
all of the children at the service settings, not just for those with disabilities. 

After participation in the training, do participants demonstrate effective performance 
skills, which facilitate prosocial behaviors and social competence in the children 
they teach? 

The Developmental Therapy Rating Inventory of Teaching Skills - Preschool Form 
(DT/RITS, Stage Two) was used by project instructors to observe and rate participants as 
they were observed working directly with children in the service setting. The instrument is 
designed to record the extent to which the suggested practices are being implemented. 
Table 3 contains post-training ratings of 64 participants at 8 sites. These results indicate 
that 84% of the participants (53) achieved an "effective" or better performance rating. The 
highest, "demonstration", level was achieved by 8% and the "effective" level, by 75%. As 
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a group, the 64 participants demonstrated proficiency skills of .80. The previously 
established criterion standard of .71 or better for an "effective" rating, indicated that the 

participants were able to demonstrate effective performance skills which had been 
specified in the inservice training to facilitate prosocial behavior and social 
competence. 

Table 3. Post-Training Performance Ratings of 64 Participants at 8 Locations 

Number of DT/RITS Ratings in Each Performance Category 









Demonstration 


Effective 


Passing 


Below Passing 








Level 


Level 


Level 


Level 


Site 




Average 


>.95 


■71 - .94 


■51 - .70 


<■50 


n 


Rating 










i 


8 


(82.1) 




8 






2 


4 


(76.0) 




3 


1 




3 


15 


(81.5) 


3 


8 


3 


1 


4 


9 


(86.2) 


1 


8 






5 


2 


(77.0) 




2 






6 


11 


(76.9) 


1 


6 


3 


1 


7 


5 


(84.0) 




5 






Other 


10 


(73.6) 


8 


2 






Total 


64 


79.9 


5 


48 


9 


2 



Concerning Children with Disabilities at the Service Settings Where the Training Occurred : 

Do the children show significant progress in social-emotional-behavioral 
development, following the inservice staff training? 

The Developmental Teaching Observation Rating Form-Revised (DTORF-R) is the 
instrument used to measure a child’s progress during the training period. DTORF-R 
training is among the first workshops provided to all sites for reliable use of this instrument. 
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DTORF-R profiles provide individual teaching objectives for each child’s social-emotional- 
behavioral development. At the conclusion of the school year, or at scheduled times 
throughout a school year, children are reassessed on the same instrument and scores 
converted to developmental age scores (in months) for statistical analysis of the gains from 
pre- to post-program. 

The staff at sites 1, 4, and 7 were able to complete both pre- and post- DTORF-R 
for children with disabilities in their inclusive programs, providing complete and reliable 
data for a total of 28 children. At the time of the first DTORF-R rating 28 children were 43.1 
months old (range = 28 - 62 months) as a group, a little over 3.5 years old. Their 
developmental age at that time was 34.9 months - a developmental lag of 8.2 months. 
When the second DTORF-R assessments were made, their teachers had participated in 
the project for 5.5 months, on average, thus the group was then 48.6 months old. The 
average post-training DTORF-R rating for the group was 47.1 , a 32.5% gain and only a 
developmental lag of 1 .5 months. This encouraging evidence was further supported by 
statistical comparison of the pre-and post-scores using a dependent / test resulting in a 
statistically significant gain ( 1 - 6.89; p< .0001). These data clearly indicated that the 
children made dramatic gains reducing their lags in social, emotional, and 
behavioral development during the time of the staff inservice training. 

To explore the question of possible differences at different sites, each set of 
DTORF-R scores was also examined separately. Table 4 contains DTORF-R data for 6 
children at site 1. As a group, these children were 50 months old at the time of the pre- 
rating (4 years 2 months) with an average developmental age score of 44 months (3 years 
8 months) representing an average 6-month lag in social-emotional development. At the 
time of the post-rating, the children were 56 months old as a group and had achieved an 
average developmental age score of 54 months, a gain of 10 months (a 23% improvement 
during the intervention period of 5.6 months on average), thus narrowing their 
developmental lag to 2 months. 
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Table 4. 

CHILD PROGRESS DATA 
Site #1 

n = 6 



ID# 


Sit 

e# 


Birthdate 


DTORF-R 
Pre-DA 
(in months) 


DTORF-R 
Post-DA 
(in months) 


CA 

Pre/Mos 


Time to 
Post 


CA 

Post/Mos 


05 


1 


02/04/92 


44 


54 


47 


4.5 


52 


06 


1 


12/16/92 


33 


48 


44 


4 


49 


07 


1 


09/30/91 


54 


59 


59 


4 


63 


08 


1 


01/13/94 


40 


48 


29 


7 


36 


09 


1 


11/23/90 


50 


62 


62 


7 


69 


10 


1 


01/30/91 


41 


51 


59 


7 


66 


Group 

Average 






44 


54 


50 


5.6 


56 


Average 3 
Groups 
Combined 






34.82 


47.07 


43.07 


5.5 


56 


Group 

Average 






35.2 


48.57 



Table 5 contains the summary DTORF-R data for 1 7 children at site 4. As a group, 
these children were 42.8 months old at the time of the pre-rating ( 3.6 years old) with an 
average developmental age score of 32.7 months representing an average 10.1 months 
lag in social-emotional-behavioral development. At the time of the post-rating, the children 
were 48.0 months old as a group and had achieved an average developmental age score 
of 47.1 months, a gain of 14.4 months (a 43.9% improvement during the intervention 
period of 5.1 months) thus narrowing their developmental lag to .9 of one month. 
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Table 5. 

CHILD PROGRESS DATA 
Site #4 
n = 17 



ID# 


Site# 


Birthdate 


DTORF-R 
Pre-DA 
(in months) 


DTORF-R 
Post-DA 
(in months) 


CA 

Pre/Mos 


Time to 
Post 


CA 

Post/Mos 


11 


4 


05/02/92 


43 


47 


42 


6 


48 


12 


4 


12/13/93 


23 


33 


33 


5 


38 


13 


4 


05/02/92 


43 


44 


42 


6 


48 


14 


4 


03/04/92 


24 


39 


47 


4 


51 


15 


4 


06/15/93 


23 


40 


33 


10.5 


44 


16 


4 


09/03/93 


18 


33 


39 


2 


41 


17 


4 


11/05/93 


21 


35 


35 


4 


39 


18 


4 


09/30/93 


9 


35 


31 


9 


40 


19 


4 


12/17/91 


41 


57 


58 


6 


64 


20 


4 


03/31/92 


66 


74 


55 


6 


61 


21 


4 


12/25/92 


15 


46 


36 


3 


39 


22 


4 


12/25/92 


40 


43 


40 


5 


45 


23 


4 


02/24/93 


43 


61 


43 


6 


49 


24 


4 


10/07/91 


24 


37 


55 


5 


60 


25 


4 


09/12/92 


40 


47 


41 


3 


44 


26 


4 


09/03/91 


50 


62 


53 


3 


56 


27 


4 


03/31/92 


33 


67 


46 


3 


49 


Group 

Average 






32.71 


47.06 


42.88 


5.09 


48.00 



Table 6 contains DTORF-R data for 5 children at site 7, a much smaller program 
and slightly younger children. Post- ratings were completed 6.8 months following their pre- 
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DTORF-R rating. As a group, these children were 35.4 months old at the time of the pre- 
rating (2.95 years old) with an average developmental age score of 31.4 months 
representing a lag of 4 months in social-emotional-behavioral development. At the time of 
the post-rating, the children were 42.2 months old as a group and had achieved an 
average developmental age score of 39.2 months, a gain of 7.8 months during the 
intervention period (a 24.8% improvement during the intervention period) thus narrowing 
their developmental lag to 3 months. 



Table 6. 

CHILD PROGRESS DATA 
Site #7 
n = 5 



ID# 


Site 

# 


Birthdate 


DTORF-R 
Pre-DA 
(in months) 


DTORF-R 
Post-DA 
(in months) 


CA 

Pre/Mos 


Time to 
Post 


CA 

Post/Mos 


28 


7 


03/26/93 


14 


29 


34 


6 


40 


29 


7 


01/05/93 


35 


50 


37 


11 


48 


30 


7 


01/15/93 


67 


52 


45 


4 


49 


31 


7 


10/10/93 


12 


22 


28 


8 


36 


32 


7 


11/20/93 


29 


43 


33 


5 


38 


Group 

Average 






31.4 


39.2 


35.4 


6.8 


42.2 



Although the developmental gains at each site varied slightly, these data support 
the conclusion that the project effect was positive across the sites with each group of 
children making notable gains and decreasing their developmental lags. 

Repeated DTORF-R measures on 5 children allowed us to examine the question 
of their rate of progress before intervention as compared with rate of progress during 
intervention. Their individual progress graphs are reported in Figures 2a - 2e. Each child 
was measured at approximately the same time intervals - approximately 3 to 4.5 months 
between each rating. The solid line indicates changes in developmental age at the 
measurement points. Four of the five children made gains at each measurement point. 
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Child #02 at Site 4 made distinct progress during the intervention, but this rate of progress 
was not as rapid as the projected rate. Program staff explained that the child was removed 
from home during this period. A traumatic reaction occurred with distinct regression in 
behavior. The dotted line indicates a hypothetical growth projection, serving as a control 
to determine if the gains were a function of program intervention or natural development. 
To obtain this projection, a rate of development prior to intervention was calculated using 
the DTORF-R developmental age score (in months) at the time of the pre- measure 
divided by the child’s chronological age (in months). This prior rate of development was 
then used to project an individual rate of development during the intervention time period 
assuming no intervention had been provided and the child continued to progress at the 
prior rate. 



Figure 2a. Individual Child Progress 

Site I. Child #1 




Solid lines = Actual rate of mastery during intervention 

Dotted lines = Calculated rate of mastery assuming no intervention 
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Figure 2b. Individual Child Progress 

Site 1. Child #2 




Figure 2c. Individual Child Progress 

Site 1. Child #3 




Solid lines = Actual rate of mastery during intervention 

Dotted lines = Calculated rate of mastery assuming no intervention 
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Figure 2d. Individual Child Progress 

Site 1. Child #4 




Figure 2e. Individual Child Progress 

Site 4. Child #25 




Solid lines = Actual rate of mastery during intervention 

Dotted lines = Calculated rate of mastery assuming no intervention 
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These figures suggest that children at the sites receiving inservice training made 
significantly greater progress during that time period than might be expected without the 
intervention. The findings also indicate that the gains were maintained four to five months 
after training ends. 

In summary, these analyses show that the effect of the project on the 
progress of children with disabilities is highly beneficial in terms of social- 
emotional-behavioral development. 

Concerning the Project as an Effective Outreach Model for Inservice Training 
Table 7 summarizes evaluations from 24 workshops including both special, topical 
sessions and those provided to developing replication sites as an aspect of the outreach 
technical assistance provided to them. Using a 5-point rating scale, with 5 being the best 
possible rating, 455 participants rated the quality of the materials presented as "very 
beneficial" (4.63), the workshop organization as "well organized" (4.63), and the overall 
impression of the workshop (4.71) as "excellent". They also reported that the workshops 
met their needs (4.46). The personal comments these participants added to the workshop 
evaluations are included as attachments. They provide a rich commentary as the positive 
effects of the workshops on participants. 

Table 7 

Cumulative Workshop Evaluations 

Number of respondents = 455 Number of workshops^24 



Average responses for numeric ratings : 



The material presented has been: 


4.63 


The workshop was: 


4.63 


My general impression of the 




workshop was: 


4.71 


This workshop met my needs: 


4.46 



Rating scale = 5 - 1 

5 1 

(Very beneficial - No benefit) 
(Well organized - Disorganized) 

(Excellent - Poor) 

(Very well - Not at all) 
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Attachment B contains a summary of a follow-up written questionnaire, mailed to 
participants at 7 replication sites. There were 19 respondents, from 5 sites. This follow-up 
was done anonymously, as several participants expressed concern to project staff about 
being able to express themselves freely at the focus groups where local 
leadership/coordinators were also participating. Project instructors concurred that this form 
of evaluation feedback might also encourage candid responses from those who had 
participated in the training. Comparison of the responses to the questions in the focus 
group (Attachment A) indicates that there was considerable similarity expressed in the two 
evaluation methods. Both focus groups and follow-up questionnaires indicated participants' 
satisfaction and included suggestions for improvement. 

To what extent do participants view the training as useful and relevant? 
Responses to a structured format in question 2 (shown in Attachment B) directly address 
this evaluation question. All 19 respondents indicated that they had used the DTORF-R for 
assessment, and all but 1 also indicated use of the DTORF-R in program planning. In 
addition, 79% used the DTORF-R for pre-post- evaluation and for selecting 
developmental^ appropriate materials; while over half reported using the DTORF-R for 
grouping children (52%). Their anecdotal responses reflected this same informal ranking 
of the elements in the training that were most helpful to them. 

Do the participants perceive the training as positive and constructive for themselves 
and the children they work with? 

Responses to question 3, (Attachment A) verify the conclusion that all of the participants 
viewed the training as positive, useful, and relevant - all of the responses to this 
question were positive. They indicated that the training affected their thinking about 
children and their own ways of working with them. Specific examples supported the 
conclusion that the basic content elements of the model were included in the training and 
are perceived by the participants to be useful and relevant. 

Questions 4, 5, and 6 provided the participants with the opportunity to evaluate 
(address) the training as it applied to their specific programs, staff, and time constraints. 
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Further, these questions enabled respondents to make suggestions that related to 
personal needs and wants for maximum successful utilization of the model. The responses 
to these questions indicate that additional training and more follow-up with direct 
observations and feedback would be beneficial. 

What suggestions do participants make for changes in the design and 
implementation of future training with this model? 

The respondents made many useful suggestions for increasing the usability of the model. 
These ideas are summarized in question 7 (Attachment A) and in question 7 (Attachment 
B). Essentially all of their ideas reflected a need for additional training with multi-media, 
real- life illustrations, and practical applications demonstrated through video and case 
materials. These recommendations are now under consideration and implementation in 
future planning as explained in the following section. 

IX. PRODUCT DEVELOPMENT AND LONG RANGE IMPACT 

This inservice training project focused to a considerable extent on a group of early 
childhood personnel which is frequently overlooked • the paraprofessional or untrained 
care giver - working in inclusive early childhood settings alongside professionally trained 
early childhood educators. They often carry a large share of responsibility for the success 
of inclusive programs of young children with disabilities as well as for those at risk. 

Because the original Developmental Therapy - Teaching model was validated for 
its effectiveness when used by teams of professionally trained teachers, the present 
project required several model adaptations, including: (1) greater focus on "talking through" 
in the training procedures for reliable use of the child-assessment instruments; (2) 
discontinuation of the use of paper-pencil tests to evaluate participants’ knowledge; (3) use 
of volunteer focus groups regarding developmental practices for evaluative project 
effectiveness feedback; and (4) preparation of training modules which provide participants 
direct with more practice and applications of specified procedures and activities. 
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This latter need was so great among participants that a compilation of activities was 
undertaken during the grant period. It is now in draft form and should be available for use 
in the field within the year. In addition, a list of several commercially available activity books 
has been prepared by the staff to aid users of the Developmental Therapy - Teaching 
model with ideas for activities to address specific developmental objectives. These 
developmentally-referenced activities should have a wide market in the early childhood 
field because of applicability for children with and without disabilities in inclusive settings. 
A commercial publisher will be sought. Several in-house audio-visual training materials 
were also revised during the grant periods, to update visuals and adapt scripts to the 
expanded training audiences. These include (1) A color slide-audio presentation of the 
steps in using the DTORF-R to assess a child’s current status in social-emotional- 
behavioral development; (2) a video cassette illustrating the unique social-emotional- 
behavioral characteristics of children at each stage of development; and (3) how adults 
adjust their own management styles to the developmental characteristics and needs of 
children with social-emotional-behavioral lags. These in-house training materials will be 
available to leadership people trained and certified as instructors by the Developmental 
Therapy Institute. 



X. STATEMENT OF FUTURE ACTIVITIES 

It is anticipated that several of the present replication sites will designate an on-site 
individual as a leadership person in the next school year. The need to have such a person 
locally was expressed repeatedly by participants in the present project. Participants 
particularly expressed the need to have such a person available to (1 ) give direct feedback 
from observations about difficult-to-manage children; (2) provide ideas for ways to enhance 
children’s program activities; (3) assist in inservice for new staff members (staff turn over 
and re-assignments are a huge problem at the local level); and (4) assist in educating staff 
of inclusive programs to the applications of the model for participating children with 
disabilities. 
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Several individuals and programs have initiated discussions with project staff to 
continue their own leadership training in response to this need. Through the 
Developmental Therapy - Teaching Programs at the University of Georgia, these potential 
leadership people will be provided with advanced training and certification as instructors. 
This network of individuals is expanding and it is anticipated that each will be able to 
generate second generation quality replications. The network will maintain connections and 
consultation with them through the Institute’s Web pages on the Internet, long-distance 
learning via satellite, newsletters, regional conferences, and via new commercial and 
professional publications. 
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ATTACHMENTS 



Attachment A: Summary of Follow-up Questionnaire: 7 Sites 

Attachment B: Summary of Focus Group Responses: Sites 1 & 2 



Attachment A 

Summary of Focus Group Responses: Sites 1 & 2 
n = 20 Participants 

(Responses to each question are summarized with most frequent responses first) 

What is the extent of your training opportunities with this project? 

• 2 days of workshops - (12 hours by project instructors) 

• 3 days of workshops - (18 hours by project instructors) 

• 20 hours (by project instructors) 

• Seminars (by a project leadership participant) 

• Parallel college course (by a project leadership participant) 

• Study of project packet of materials 

• Observation of child assessment procedure 

Describe your use of aspects of the training. 

• Use of the assessment procedure 

• Recognizing developmental milestones in children’s behavior 

• Planning for developmental milestones on curriculum activities 

• Understanding why certain strategies should be used 

• Understanding why children do what they do (decoding behavior) 

• More feeling for what child is feeling 

• How to manage problems with effective, positive strategies: e.g. 

Using proximity/modeling/eye-level contact/body 
language/adult interaction (lead & support roles) 

Affirming children’s behavior/affirming their feelings 

• Making positive instead of negative statements 

• Structuring choices for children 

• Using "easy, supportive touch" 

• "like a painting of their developmental mind grade" 

How could this model be used fully in your own setting? 

• Assessing all children (those without disabilities) to design individual objectives 

& program 

• Increasing amount & frequency of parent-teacher staffings 

• Planning curriculum 

• Meeting individual children’s needs 

• Integrating our children with special needs into regular classrooms 

• Grouping breakout groups for specific needs and objectives 

• Expanding parents’ understanding of children’s needs at home & school 

• Expanding parents' understanding of children’s program objectives 

• Orienting to individual needs and objectives rather than by labels 

• Recognizing individual rates of development 

• Recognizing administrative support for teachers in the program 



• Providing more follow-through in inclusive setting with regular staff 

• Using the color-coded bar graph of a child’s progress 

• Providing more staff time for assessing more children 

• Providing someone else to do the paper work 

• Changing mind sets about discipline & power struggles between some teachers 

& children 

How has the model/project affected your thinking about how you work with 
children -- or your actual work? 

• Understanding of children’s patterns, rates, & sequences of development 

• Ways to put developmental concepts into the program & curriculum 

• Ways to group in classes 

• Helps to guide children’s progress developmental^ 

• Support & supervision of classroom personnel (from leadership participant) 

• Understanding goal setting & individual program variation (from leadership 

participant) 

• Making strategic decisions for individual program direction 

Describe things which may have limited your use of the model, or been a 
constraint. 

• Children’s repeated assessment schedule was too limited 

(Assessments need to be more frequent for better understanding of a child) 

• Time limitations for child assessments and individual team planning, especially 

with parents 

• Monitor actual gains more closely so that corresponding program changes can 

be made 

• Local & state program policies (requirements for age groupings and staff ratios) 

are not sufficiently flexible for program needs of individual children 

• The assessment questions (DTORF-R) are too hard 

• They (DTORF-R) need to be written so people can understand them without 

reading "every little bit of it" 

• "The list of kids that need to be assessed is too long" 

• "It tells you what to do and what not to do" 

• Its hard to stay on the planned assessment schedule 

• Its difficult to involve parents and staff in the other (regular) classrooms 

What needs do you still have about the use of the model, which could be 
constraints also? 

• More time is needed for child assessments and team planning 

• A collection of suggestions/activities for meeting individual objectives (in the 

group) 

• More resource materials to address individual & group objectives 

• More feedback is needed about reliability in using the model 

• A printout of each developmental profile/rating for parents 



• A guide for parents to use similar, positive management strategies 

• We have too many kids for recommended size of groups 

• Time is needed for teachers to have individual breaks 

• One staff person is needed to go from room-to-room for feedback and resource- 

ideas 

How can we make the training experience more helpful? 

• Provide a resource book of activities to meet specific objectives 

• We need a way to share ideas & activities 

• Video examples of management strategies would be helpful 

• Use video for illustrating the steps in rating with the DTORF-R 

• The inservice training needs to be longer (e.g., 7 weeks) 

• Space out the training and focus in more detail 

• Include homework and planning actual classroom activities 

• Provide a CD ROM with suggestions about ways to work on the objectives 

• Give detail on ways to work on the same objective with children of different ages 

• Use a famous movie like "Tom Sawyer" to practice DTORF-R ratings 

• Have everyone practice DTORF-R ratings on the same child and then follow up 

with discussions 



Attachment B. 

Summary of Follow-up Questionnaire: 7 Sites 

19 respondents 

(Responses to each question are summarized with most frequent responses first) 

1. What is the extent of your training opportunities with this project? 

(13 responses) 

• 1 day workshop - (4-6 hours by project instructors) 

• 2 days of workshop - (12 hours by project instructors) 

• 3 days of workshop - (18 hours by project instructors) 

• Observation of child assessment procedure 

• 3-5 seminars (by project instructors) 

• 20 hours (by project instructors) 

• Seminar (by a project leadership participant) 

• Parallel college course (by a project leadership participant) 

• Study of project packet of materials 

• Observation of child assessment procedure 

• Observation/visit by staff to classroom 

2. Which components of the model have you used? 

• DTORF-R assessment procedure (15 responses) 

• DTORF-R results for program planning (14 responses) 

• Selecting developmental appropriate materials (13 responses) 

• DTORF-R results for grouping children (1 1 responses) 

• Planning developmental^ schedules (10 responses) 

• Understanding team roles (10 responses) 

• DTORF-R pre- post- evaluation (9 responses) 

• Understanding why children do what they do (decoding behavior) (8 responses) 

• Using positive management strategies (6 responses) 

Which ones have you found most helpful? 

• The DTORF-R for assessment and understanding developmental differences 
(9 responses) 

• Developmental^ appropriate schedules, material, & activities (6 responses) 

• Individual program planning (5 responses) 

• Team work and team roles (4 responses) 

• Grouping children for program planning (3 responses) 

• Decoding behavior/understanding/empathy (3 responses) 

• Positive management strategies (2 responses) 

3. Has Developmental Therapy-Developmental Teaching affected your thinking 
or beliefs about working with children? 

(All 15 answered in the positive) 



In what ways? 

• Meeting individual children’s needs 

• Designing individual programs 

• Planning curriculum 

• Recognizing individual rates of development 

• Understanding of children’s patterns, rates, & sequences of development 

• Way to group children 

• Ways to provide developmental^ appropriate structure 

• Understanding different needs individual children have 

• Additional techniques for working with children 

• Using reflection and other positive strategies to build bond/relationships with 

children 

Affected your actual work with children? 

(All 1 4 were positive responses, with 8 "yes") 

• Understanding different needs of individual children/different goals 

• Using different strategies to meet different individual needs 

• Assessing children and using the scoring to plan programs; different strategies 

to meet different individual needs 

Describe any things which may have limited your use of the model. 

(14 responses) 

• A lot of information in 2 to 3 days 

• Assessing children who are in our program less than 6 months 

• Time limitations for doing child assessments as frequently as desirable 

• Providing more staff time for assessing children 

• Providing more staff time for planning 

• Staff turnover, with time between training too long to get new staff up to speed 

• Frequent reviews would help recalling best practices to use 

• More training needed 

• More time needed to implement exemplary practices 

• DTORF-R is not written in easy-to-understand language 

Explain any needs you may still have about using the model? 

(13 responses) 

• Suggestions of activities for meeting individual objectives (in the group) 

• More training 

• Annual refresher course 

• More information about DTORF-R group ratings, overtime, and calculating 

percentages 

• Additional follow-up in the classroom by project instructors 

• no additional needs 

• Immediate training for new staff members 

• Real life examples of healthy intervention strategies 



6. How can we make the training experience more helpful? 

(16 responses) 

• Provide more observing and feedback 

• More training on the DTORF-R 

• Information about when refresher training will be offered 

• Additional focus on implementing in the classroom 

• More training in specific activities to meet objectives 

• A training video 

• Train an experienced teacher (on site) to teach new staff 

• Role play how to use (methods) 

• Emphasize that parents do not have to be present (to do the assessment) 

• Use specific cases to illustrate applications & examples 

7. How can we increase the usability of the model? 

(14 responses) 

• Provide books & videos to use as reference at local site 

• Provide a trainer to assist in assessments, problem-solve & general 

implementation at local site 

• Give more detail on ways to work on the same objective with children of different 

ages 

• Provide a computer disk to input information 

• Video conferences to train new staff & update others 

• Model is very usable! 

• Provide specific suggestions and in-depth on tactics to meet specific objectives 

• Demonstrate with real life situations or examples 

8. Are you more likely to include children with social-emotional-behavioral 
delays in your program following training? 

(16 responses) 

• 10 responses that they already have a considerable number of children with such 

disabilities 

• 6 "yes" responses 
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